Contactless operation of devices using a pointing apparatus

ABSTRACT

Contactless operation of a device is provided via a mobile pointing apparatus and a receiving arrangement associated with the device. The mobile pointing apparatus includes a signal emitter for emitting an optical or electromagnetic signal, and the receiving arrangement associated with the device determines a pointing target of the mobile pointing apparatus relative to the device based on the signal emitted by the mobile pointing apparatus and triggers a function of the device based on the pointing target of the mobile pointing apparatus.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.17/677,069, filed on Feb. 22, 2022, which claims priority to EuropeanPatent Application No. EP 21167557.4, filed on Apr. 9, 2021. The entiredisclosures of both of the aforementioned applications are herebyincorporated by reference herein.

TECHNICAL FIELD

Exemplary embodiments of the present invention relate to apparatuses andmethods for the contactless operation of medical devices.

BACKGROUND

Special hygiene standards apply to medical devices in order not toendanger the patients treated with these devices by transmittingpathogens. In everyday clinical practice, such devices are operated bydifferent medical staff. The same staff is also involved in actions ondifferent patients. There is a risk here that pathogens such as virusesand bacteria, irrespective of their source (patient, staff, visitors,etc.), are transmitted to patients and/or medical staff via theoperating surfaces of the medical devices to which they have beenapplied by touch, thus endangering the health of persons, especially ofpatients who are already impaired.

To prevent (cross-)contamination via such devices, extensive hygienemeasures are usually taken. These consist in particular in the wearingof disposable gloves by medical staff, which are discarded after anaction on the patient and/or a medical device interacting with thispatient and replaced by new sterile gloves. This constant changing ofgloves is expensive and time-consuming, especially in clinics, intensivecare units or dialysis centers. Furthermore, there is a need for a highhand disinfection rate among medical staff. This measure is alsoexpensive and time-consuming due to the necessary use of disinfectantsand is potentially harmful to the skin of the staff. In addition, thereis also a need to disinfect medical devices frequently. Touchscreens inparticular need to be disinfected very frequently. In addition to thedisadvantages mentioned above, this causes damage to the touchscreenover time due to the disinfectants used and the mechanical wear causedby the actual disinfection activity due to, for example, the wipingmotion with cloths during disinfection.

There is therefore a desire to avoid touching medical devices as much aspossible. In the state of the art, voice control and gesture controlmethods are known for this purpose. However, both methods are not veryuser-friendly and intuitive in everyday clinical use. Voice control isproblematic in busy environments with background noise and is alsodisadvantageous with regard to the private patient-related character ofthe information entered. Gesture control requires extensive stafftraining and is often cumbersome and ill-suited to the dynamic dailyroutine within medical facilities.

From U.S. Pat. No. 8,570,274 B1 it is known to use a pointing stickapparatus for contactless control of a user interface in a personalcomputer. In the embodiment described in detail, the pointing stickapparatus communicates with receivers installed on the laptop viaultrasound. However, this device is of only very limited use for therequirements characteristic of medical technology mentioned at theoutset.

SUMMARY

In an exemplary embodiment, the present invention provides a system forcontactless operation of a medical device. The system includes: anoptical pointing apparatus, comprising an optical signal emitterconfigured to emit a directional light signal; one or more opticalsensors configured to determine a pointing target of the opticalpointing apparatus relative to the medical device based on thedirectional light signal emitted by the optical pointing apparatus; anda processor configured to trigger a function of the medical device basedon the determined pointing target of the optical pointing apparatus.

In a further exemplary embodiment, the medical device comprises ahemodialysis device, a peritoneal dialysis device, an infusion pump, amedical monitoring apparatus, an electrocardiogram (ECG) device, amedical ultrasound device, a cell separation apparatus, or a heart/lungmachine.

In a further exemplary embodiment, the directional light signal is alaser signal.

In a further exemplary embodiment, the one or more optical sensorscomprises one or more phototransistors and/or one or more photodiodes.

In a further exemplary embodiment, at least a part of the medical devicecomprises a coating for redirecting the directional light signal emittedby the optical pointing apparatus to the one or more optical sensors.

In a further exemplary embodiment, the medical device comprisesoperating components and a transparent cover for the operatingcomponents, wherein the coating is at least partially disposed on thecover.

In a further exemplary embodiment, the optical signal emitter isconfigured to emit light in the non-visible spectrum, and the opticalpointing apparatus further comprises a second optical signal emitterconfigured to emit light in the visible spectrum.

In a further exemplary embodiment, the optical pointing apparatusfurther comprises: at least one push-button; and a modulator configuredto modulate information on the directional light signal emitted by theoptical signal emitter in response to actuation of the at least onepush-button.

In a further exemplary embodiment, the optical pointing apparatusfurther comprises a fingerprint scanner and/or a communication deviceconfigured to communicate with a fingerprint sensor wearable under aglove of an operator.

In a further exemplary embodiment, the one or more optical sensors arearranged at the medical device or arranged remotely relative to themedical device with a defined spatial relationship thereto.

In another exemplary embodiment, the present invention provides a systemfor contactless operation of a medical device. The system includes: anelectromagnetic pointing apparatus, comprising at least oneelectromagnetic signal emitter configured to emit a radar signal; atleast three receivers configured to receive the radar signal emitted bythe at least one electromagnetic signal emitter; and a processorconfigured to: determine a pointing target of the electromagneticpointing apparatus relative to the medical device based on the receivedradar signal; and trigger a function of the medical device based on thedetermined pointing target of the optical pointing apparatus.

In a further exemplary embodiment, the processor is further configuredto compare the radar signal with a reference signal.

In a further exemplary embodiment, the system further includes: anelectromagnetic signal emitter associated with the medical deviceconfigured to emit a trigger signal to cause the electromagneticpointing apparatus to emit the radar signal from the at least oneelectromagnetic signal emitter.

In a further exemplary embodiment, the processor is configured toperform a frequency modulated continuous wave (FMCW) method fordetermining a distance between the at least one electromagnetic signalemitter of the electromagnetic pointing apparatus and the at least threereceivers.

In a further exemplary embodiment, the electromagnetic pointingapparatus further comprises at least a second signal emitter configuredto emit a second radar signal and/or a sensor for determining aninclination of the electromagnetic pointing apparatus, and theelectromagnetic pointing apparatus is further configured to determine apointing axis of the electromagnetic pointing apparatus.

In a further exemplary embodiment, the electromagnetic pointingapparatus further comprises: at least one push-button; and a modulatorconfigured to modulate information on the radar signal emitted by the atleast one electromagnetic signal emitter in response to actuation of theat least one push-button.

In a further exemplary embodiment, the electromagnetic pointingapparatus further comprises a fingerprint scanner and/or a communicationdevice configured to communicate with a fingerprint sensor wearableunder a glove of an operator.

In a further exemplary embodiment, the at least three receivers arearranged at the medical device or arranged remotely relative to themedical device with a defined spatial relationship thereto.

In yet another exemplary embodiment, the present invention provides amethod for contactless operation of a medical device. The methodincludes: emitting, by an optical signal emitter of an optical pointingapparatus or at least one electromagnetic signal emitter of anelectromagnetic pointing apparatus, a directional light signal or aradar signal; determining, using one or more optical sensors or at leastthree receivers, a pointing target of the optical pointing apparatus orthe electromagnetic pointing apparatus relative to the medical devicebased on the directional light signal or the radar signal; andtriggering, by a processor, a function of the medical device in responseto the determined pointing target.

BRIEF DESCRIPTION OF THE DRAWINGS

Exemplary embodiments of the present invention will be described in evengreater detail below based on the exemplary figures. The invention isnot limited to the exemplary embodiments. All features described and/orillustrated herein can be used alone or combined in differentcombinations in embodiments of the invention. Features and advantages ofvarious embodiments of the present invention will become apparent byreading the following detailed description with reference to theattached drawings which illustrate the following:

FIG. 1 depicts a relationship between emit and receive signals in thedistance determination with a frequency modulated continuous wave (FMCW)radar;

FIG. 2 depicts projection of the pointing pen axis onto the surface ofthe device according to embodiments of the invention;

FIG. 3 depicts position determination of the pointing pen with thepointing pen signal triggered by a trigger signal according toembodiments of the invention;

FIG. 4 depicts encoding of information in the pointing pen signal bycharacteristic interruption of the signal according to embodiments ofthe invention;

FIG. 5 depicts encoding of information in the pointing pen signal byappending the information to a modulation period in accordance withembodiments of the invention; and

FIG. 6 depicts determining the inclination of the pointing pen withangular or inclination sensors according to embodiments of theinvention.

DETAILED DESCRIPTION

Exemplary embodiments of the present invention provide apparatuses andmethods that allow improved contactless operation of medical devices.

Exemplary embodiments of the invention relate to a system forcontactless operation of a device. The system comprises a mobilepointing apparatus, in particular in the form of a pen, and a receivingarrangement (e.g., a receiving system) associated with the device.

The device may in particular be a medical device, i.e. any type ofdevice used in medical technology, for example a hemodialysis device, aperitoneal dialysis device, an infusion pump, a medical monitoringapparatus (such as, for example, an electrocardiogram (ECG) device), amedical ultrasound device, a cell separation apparatus, a heart/lungmachine, or any other medical device which utilizes regular operatorinput during operation.

The mobile pointing apparatus may comprise a signal emitter for emittinga signal. The receiving arrangement associated with the device maycomprise means (e.g., a signal receiver) for determining a pointingtarget of the mobile pointing apparatus relative to the medical devicebased on the signal emitted by the mobile pointing apparatus, and means(e.g., a processor) for triggering a function of the device inaccordance with the pointing target of the mobile pointing apparatus.

Thus, the operation of the (medical) device can be completelycontactless. For example, an operator can point the mobile pointingapparatus at certain parts of the (medical device) (e.g. an on/offbutton) and thereby trigger a corresponding function on the device (e.g.switching on or off). Further exemplary functions include, for example,requesting operating parameters of components of the (medical) device,such as the delivery speed of a pump for a medical fluid or the volumeof a fluid container, adjusting a component of the (medical) device,such as the delivery speed of a pump for a medical fluid, interactingwith a graphical user interface (GUI) of the (medical) device, and soon. Furthermore, a much more complex functional operation may beachieved, which will be further explained below.

In one possible embodiment, the signal emitter may comprise an opticalsignal emitter for emitting a directional light signal, in particular alaser signal. The means for determining the pointing target of themobile pointing apparatus may comprise one or more optical sensors, inparticular phototransistors and/or photodiodes. Such an embodiment maybe based on a laser pen.

Part of the device may have a coating, in particular a PMMA(polymethylmetacrylate) coating, for redirecting the light signalemitted by the mobile pointing apparatus to the one or more opticalsensors. This increases flexibility with regard to the placement of thesensors.

The device can have operating components and a transparent cover for theoperating components, whereby the coating is at least partially arrangedon the cover. Thus, devices located in housings or behind covers canalso be operated, which has hygienic as well as optical reasons.

In another aspect of the invention, the optical signal emitter may beconfigured to emit light in the non-visible spectrum. The mobilepointing apparatus may further comprise a second optical signal emitterfor emitting light in the visible spectrum. This makes the operationparticularly robust versus ambient light, while still being able toprovide optical feedback.

As an alternative to the laser embodiment explained above, the signalemitter may also comprise a signal emitter for emitting anelectromagnetic signal, in particular a radar signal. The means fordetermining the pointing target of the mobile pointing apparatus maythen comprise at least three receivers for receiving the electromagneticsignal emitted by the signal emitter and means (e.g., a processor) forcomparing the electromagnetic signal with a reference signal, preferablystored in the device. In this embodiment, the signal communication isvery robust, which leads to a particularly reliable operability.

The receiving arrangement associated with the device may comprise anelectromagnetic signal emitter for emitting a trigger signal suitablefor causing the mobile pointing apparatus to emit the electromagneticsignal from the signal emitter. Thus, for the determination of theposition of the mobile pointing apparatus, reflected radar signals arenot evaluated, as is conventionally the case, but signals emitteddirectly by the pointing pen. This allows a particularly stabledetection of the signals emitted by the pointing pen. In this case, themeans for determining the pointing target of the mobile pointingapparatus are preferably configured to perform a FMCW (frequencymodulated continuous wave) radar method for determining a distancebetween the mobile pointing apparatus and the receiving arrangementassociated with the device.

Also, the mobile pointing apparatus may further comprise means (e.g., apointing axis detection system) for determining a pointing axis of themobile pointing apparatus. This may be a second signal emitter foremitting an electromagnetic signal, in particular a radar signal, and/ora sensor for determining an inclination of the mobile pointingapparatus, in particular an angle sensor and/or an inclination sensor.

Regardless of the selected type of signal transmission (e.g. laser orradar), the mobile pointing apparatus may further comprise at least onecontrol element, in particular at least one push-button, as well asmeans (e.g., a modulator) for modulating information on the optical orelectromagnetic signal emitted by the signal emitter in response to anactuation of the at least one control element. Thus, in principle, anyinformation can be transmitted to the device, e.g. control commands.

The mobile pointing apparatus may further comprise means (e.g., anidentification system) for identifying an operator. These means maycomprise a fingerprint scanner or means (e.g., a communication device)for communicating with a fingerprint sensor wearable under a glove ofthe operator. This ensures secure identification of the operator priorto operation, which counteracts misuse.

The means for determining the pointing target of the mobile pointingapparatus may be connected to the device, in particular arranged on thedevice. Alternatively, the means for determining the pointing target ofthe mobile pointing apparatus may be arranged remotely from the devicewith a defined spatial reference to it, for example at predefinedpositions in a room. This makes it possible, among other things, to makeseveral devices operable at the same time.

Furthermore, exemplary embodiments of the invention also relate to theindividual components explained above in their own right, namely amobile pointing apparatus, a (medical) device, and/or a receivingarrangement associated with the (medical) device for use in a system asdescribed above. Thereby, these apparatuses have the features of therespective apparatus as described above.

Also provided is a method for contactless operation of a device,comprising emitting a signal by a signal emitter of a mobile pointingapparatus, in particular in the form of a pen, determining a pointingtarget of the mobile pointing apparatus relative to the device based onthe signal emitted by the mobile pointing apparatus by a receivingarrangement associated with the device, and triggering a function of thedevice in dependence with the pointing target of the mobile pointingapparatus by the receiving arrangement associated with the device. Inthe method, the mobile pointing apparatus, the device and/or thereceiving arrangement associated with the device may be any of theapparatuses described above.

Finally, the invention provides a computer program with instructionsthat cause the mobile pointing apparatus, the (medical) device and/orthe receiving arrangement associated with the medical device to performthe respective method steps explained herein.

It will be appreciated that the execution of the variousmachine-implemented processes and steps described herein may occur viathe execution, by one or more respective processors, ofprocessor-executable instructions stored on a tangible, non-transitorycomputer-readable medium, such as random access memory (RAM), read-onlymemory (ROM), programmable read-only memory (PROM), and/or anotherelectronic memory mechanism. Thus, for example, operations performed bya medical device, a medical system, or other devices as discussed hereinmay be carried out according to instructions stored on and/orapplications installed on one or more respective computing devices.

In the following, exemplary embodiments of a system according to theinvention for the contactless operation of a medical device, as well asits individual components and corresponding methods, are explained inmore detail.

Exemplary embodiments of the present invention provide intuitive controlof devices by evaluating a pointing or gesturing movement towards thecomponent of a medical device to be operated and/or towards a surface ofa graphical user interface (GUI) thereof.

According to the invention, a mobile or portable pointing apparatus, inparticular a pointing pen (PEN), can be used for this purpose. In thefollowing, the principles of the present invention are explained withreference to a pen for the sake of simplicity; however, it is understoodthat these principles are equally implementable with other form factors.The position of the preferred longitudinal axis of the pen in space isdetermined. By computational (e.g. vector analysis, geometriccalculations) extension of this axis to the surface of the device to beoperated, the point of impact of the extension of the determined penaxis on the device surface can be determined. An appropriatelyconfigured control device links the point of impact with predefinedactions, in particular with the triggering of a function of the medicaldevice. If, for example, it is determined that the point of impact iswithin the area of a mounted blood pump, e.g. of a dialysis machine, itcan be provided that a setting menu for this pump (e.g. for pump rate,start, stop, etc.) is displayed on the GUI. In this way, an operator canselect components for operation only by pointing to a device with anappropriately configured pen. This also includes selecting areas on ascreen. In this way, touchscreens can be avoided, for example.

To determine the pen axis, the pen is equipped with at least one signalemitter according to exemplary embodiments of the invention.

Embodiment with Laser Emitter

In this embodiment, the mobile pointing apparatus has one, preferablyonly one, signal emitter in the form of a directed light source, forexample in the form of a laser. Analogous to a laser pen, an operatorcan thus point visibly to components of a device to be operated in orderto select them. For this purpose, the device which is to be operatedpreferably has optical sensors (e.g. phototransistors and/orphotodiodes) at corresponding points which can be targeted by anoperator.

The sensors can be identified optically (e.g. by outlines or coloring).

In this embodiment with an optical emitter, it is not necessary todetermine the position of the preferred longitudinal axis of the pen inspace, since the point of impact on the surface of the device to beoperated is determined by corresponding optical sensors on the deviceitself, independently of the position of the pen in space.

In an exemplary embodiment, the pen may be equipped with two lightsources that radiate light in (narrow) parallel axes, one of whichradiates light in the non-visible spectrum to which the optical sensorsof the device are sensitive (e.g. UV laser and visible red). In thisway, the operation can be made robust versus ambient light and stillprovide optical feedback (aiming aid).

A coating can also be applied that redirects optical radiation todistributed optical sensors, for example as explained in: DonatoPasquariello, M. C. J. M. Vissenberg, and Galileo June Destura,“Remote-Touch: A Laser Input User-Display Interaction Technology” J.Display Technol. 4, 39-46 (2008). Such a coating may include atransparent microstructured light-guiding substrate made of polymethylmethacrylate (PMMA). The substrate is structured with a microstructureof pyramid-shaped depressions. When the laser light hits the substrate,it is collected by the microstructure in the PMMA substrate and part ofthe laser light is reflected to the side parallel to the surface andthus redirected to optical sensors located there, which can thusdetermine the position of the laser spot. Such an apparatus comprising atransparent PMMA coating present on the surface of a screen can providethe functionality of a touchscreen without the need for touching thescreen. In addition, inexpensive and scalable screens can thus beimplemented, since, especially for large surfaces, the number of sensorsutilized can be significantly reduced.

The same principle of optical redirection can be applied to the entirebody of the device. Thus, it is possible to equip an opticallytransparent cover in front of operating components of a device with theabove-mentioned PMMA coating, which intercepts occurring laser radiationand redirects it to optical sensors. In this way, the operator cantarget components located behind the cover for selection. An example ofa medical device having an optically transparent cover is FreseniusMedical Care's 5008 hemodialysis machine. This is equipped with twotransparent hinged doors which cover the extracorporeal blood circuitbehind them during operation. This is not only for optical reasons butalso for reasons of hygiene.

In addition to the selection of components, input of information for theoperation of a medical device is also provided. Conventionally, this isrealized, for example, via input elements such as switches or buttons ona touchscreen display. In embodiments of the invention, this is achievedwith the pointing pen in a contactless manner by the pointing pen havingat least one input option, for example a push-button, which can beactivated by the operator. The activation has the effect that theemitted light signal is modulated with information. In one example,pulse code modulation may be used, whereby a pulse code switches thesignal on and off in an analog manner so as to create a characteristicdigital sequence which encodes the information to be transmittedAmplitude modulation may also be used.

The receivers on the medical device (these do not necessarily have to bearranged directly on the device but can also be assigned to it at adistance; see below) are set up to demodulate the light signal. Themedical device carries out steps according to the received information.

For example, the transmitted information can be characteristic for thepointing pen. Accordingly, the medical device can allow, deny and/orrestrict operability. Accordingly, the user can also identifyhimself/herself on the pointing pen. Here, for example, the pointing penmay be provided with a fingerprint scanner for authentication, and anidentification signal linked to a stored fingerprint may be modulatedonto the light signal.

Identification features can, but do not have to be, sent only afteractivation of the push-button. They can also be sent permanently (i.e.periodically without an activation signal).

The authentication of a user at the pointing pen can be carried out invarious ways, for example via a PIN code entry. Alternatively oradditionally, in the context of multi-factor authentication, theauthentication of the user can be performed via additional peripheraldevices. The peripheral device may be carried by the user, may be partof the pointing apparatus or medical device, or may be in data exchangewith the pointing apparatus or medical device. For example, it is awristband, a smartwatch, a smartcard, or a sensor for capturingbiometric characteristics of the user.

In another embodiment, a fingerprint sensor (such as marketed as a MicroRF Tag by SK-Electronics Co Ltd) may be used that can be worn under theglove and transmits authentication data via wireless connection (e.g.NFC) to the pointing pen as soon as the hand of the operator comes closeto the pointing pen. This ensures, among other things, that a stolenpointing pen is unusable.

The fingerprint sensor preferably contains a two-dimensional arrangementof the above-mentioned passive RFID tags in the dimensions of afingerprint or a unique fingerprint portion. The fine segmentation bythe small tags enables the detection of the fingerprint-defining lines(papillary strips). For this purpose, the sensor is applied to theunderside of the fingertip (pressing on by glove and/or self-adhesive).The emission behavior of the individual tags is determined by therespective substrate and its electrical conductivity. The contact ornon-contact with the papillary strips creates a characteristicconductivity distribution that leads to the corresponding emissionbehavior of the individual tags. To control the conductivity, a pastewith high wettability between the sensor and the finger can beadvantageous; this paste can also contribute adhesively to theattachment of the sensor. If this paste is heat-dependent, especially ifit has phase transitions around skin surface temperatures, it can beensured that the scanned finger is warm, usually that of a real (warm)person, to make misuse more difficult. A gallium-based paste isadvantageous in terms of the achievable melting point, electricalconductivity and high wettability, which lead to accurate imaging of thefingerprint. In this embodiment, the pointing pen comprises means (e.g.,a communication device, such as an RFID reader) for communication withthe sensor. In this way, the pointing pen can read the signals from theemitting tags of the sensor through the glove worn by an operator. Fromthis, the fingerprint can be reconstructed and subsequently used forauthentication.

The actuation of the push-button as an input means can transmit a codethat can be interpreted as the actuation of a selection means. Forexample, if the blood pump of a hemodialysis device is selected with thepointing pen, a selection menu for setting this pump can be displayed onthe screen. This can include a simple “more” or “less”, and/or may alsoinclude displaying a number field with a confirmation field. Theoperator can select the selection fields on the display in the manneralready described and activate them by pressing the push-button. In thisway, the operator can make entries on the medical device and thusoperate it completely without touching it.

Embodiment with Two Radar Emitters

In addition to optical signals, other electromagnetic signals can alsobe used. Electromagnetic signals with small wavelengths such as radar,which can detect the smallest movement amplitudes, are advantageous.Radar sensor technology includes signal generators and receivers, eachin the form of antennas or receiver coils.

In the following, one embodiment of the invention will be explainedusing the example of radar for electromagnetic waves. However, it willbe apparent to those skilled in the art that other electromagnetic wavescan also be used, or that the term radar can be broadly defined withrespect to the wavelength range.

In the state of the art, radar sensor technology is regularly used fordistance and motion detection, in which an emitted signal is evaluatedin relation to a signal reflected by an object with regard to signalamplitude and phase relationship (temporal offset). A well-knownapplication is the speed monitoring of vehicles. Recently, however,radar sensor technology has also been commercially offered that allowsevaluations of the smallest movement amplitudes and can thus also beused to measure vital parameters of patients (see, for example, WO18/167777 A1). The corresponding emitting and receiving units aresufficiently compact to be installed in a medical device in largenumbers. One field of application of radar sensor technology ishigh-precision distance measurement.

To determine the distance to an object, for example, the principle offrequency modulated continuous wave radar (FMCW radar) can be used. Thisdistance measurement includes the comparison of a reflected radar signalwith a reference, usually a radiated radar signal. The distance can bedetermined via the propagation speed of radar waves (=c₀ speed of light)and the transit time between radiation and reception of the reflection.Due to the very high propagation speed and the relatively close distancein industrial applications, the direct determination of the propagationtime is technically difficult. A way out is offered by theabove-mentioned FMCW method, which uses a continuously frequency-varyingsignal as the excitation signal and instead of the propagation time usesthe frequency difference between the exciting and reflected signal tomeasure the distance, which is technically more precise and less complexto implement.

In FMCW, the distance between the emitter/receiver and the measuredobject is determined according to the following relationship:

$R = {\frac{\left. c_{0} \middle| {\Delta t} \right|}{2} = \frac{\left. c_{0} \middle| {\Delta f} \right|}{2\left( \frac{d(f)}{d(t)} \right)}}$

wherein c₀=speed of light=3×10^(8m)/_(s); Δt=propagation time [s];Δf=measured frequency difference [Hz]; R=Distance antenna-object [m];d(f)/d(t)=Frequency sweep per time unit.

Here, the frequency sweep per time unit is the slope of the frequencymodulation, as shown in FIG. 1 . The signal shown solid is anemission/reference signal and the signal shown dashed is a reflectedsignal.

Examples of radar-based distance measurement can be found in DE102019002278A1, WO 2016/204641A1, WO 2020/127177A1, WO 9908128A1 and WO07009833 A1.

Here, the FMCW measurement represents only an exemplary embodiment.

In order to make the above-described principle of contactless controlalso implementable with radar sensor technology, the medical device inone embodiment has at least three receivers (for determining thetransmitter positions in space) sensitive to radar signals (orelectromagnetic signals of specific wavelengths) and/or at least oneemitter for an electromagnetic signal. Furthermore, the pointing pen hasat least one emitter, as well as means for determining the position ofthe pointing axis with respect to the emitter. In one embodiment, thismeans is at least one further second emitter spatially remote from thefirst emitter. In a further embodiment, the pointing pen also comprisesa receiver. In all embodiments, the pointing pen comprises a controllerdevice and a power source. In further embodiments, the emitter alsocomprises input means for inputting information, analogous to theembodiments with laser emitters described further above.

Preferably, the embodiments of the invention based on the evaluation ofelectromagnetic beams are based on the determination of the position ofthe axis of the pointing pen with respect to the device that is to beoperated and, derived therefrom, the determination of the position ofthe component of the medical device that is targeted by the operator,which can be determined by projecting the axis thus determined onto thesurface of the device. This is illustrated in FIG. 2 .

In this example, the medical device has four receivers (E1-E4). Anemitter on the medical device is not shown here. The pointing pen hastwo emitter S1 and S2, which are arranged on the longitudinal axis ofthe stick. Via the method described below, the distance of the emittersS1 and S2 to each of the receivers can be determined (indicated by thesolid and dashed lines from the receivers to the emitters) and thus theposition of the pointing pen to the medical device can be determined bypurely geometric calculations in the medical device. Thus, theprojection of the longitudinal axis onto the surface of the device(arrow to A) can also be determined. The position in the exampleaccording to FIG. 2 results from the fact that the distances of theemitters S1 and S2 to the receivers E1-E4 can be determined and thusgeometrically unambiguous pyramids result, the bases of which are formedby the rectangle at the corners of which the receivers E1-E4 are locatedand the respective tips of which are formed by the emitters S1 and S2.The position of the tips is defined by the distances to the individualreceivers. The length of the pyramid edges corresponds exactly to therespective defined distance. Thus, for example, the coordinates of theemitters S1 and S2 can be determined in a reference space. With thesecoordinates, a vector can be determined on which the emitters S1 and S2lie. By mathematically determining the intersection of this connectionvector with the plane in the reference space on which the receiversE1-E4 lie, the point of impact A can be determined. Four receivers areshown in FIG. 2 . At least three receivers are utilized to determine thepoint of impact.

In embodiments of the present invention, in order to obtain stablesignals emitted by the pointing pen, no reflected radar signals areevaluated, but signals emitted directly by a pointing pen are evaluatedin order to determine the position of the axis of the pointing pen bycorresponding signal evaluation. To determine the distance to the pointof emission, the signal emitted by the pointing pen behaves like areflected signal. To achieve this, trigger signals are emitted from themedical device which, when received by the pointing pen, initiate theemission of the radar signals from the two emitters.

Referring to the FMCW embodiment and FIG. 3 , this method comprises thefollowing steps:

At time tS, the emitting apparatus emits an (arbitrary) electromagnetictrigger signal. The time tS is associated with the signal shown solid.The signal shown solid is only available as a reference signal path inthe medical device and is not emitted. The signal shown in dashed linesis compared with this stored reference signal, e.g. the differencefrequency is determined, from which the distance between emitter andreceiver can be derived. The trigger signal can be emitted periodicallyto enable permanent position determination.

At the time (tS+tE)/2, the receiving apparatus of the pointing penreceives the trigger signal and immediately initiates the emission ofthe characteristic dashed signal, which is detected at the time tE atone of the receivers of the medical device. Via the comparison describedabove, the distance between the emitter and the respective receiver canbe determined accordingly.

The triggering process may also be triggered by the pointing pen, forexample by a movement detection in the pointing pen detecting amovement, emitting an activation signal via (any) emitter, which isreceived by the medical device, whereupon the medical device emits thetrigger signal in the manner described above. In this way, no permanenttriggering occurs, but only when movement of the pointing pen isdetected.

It is not essential that emitter and receiver are arranged in the sameplace on the medical device. Consequently, a single emitter can be usedon the medical device, which advantageously has the same distance to allreceivers on the medical device. However, it is equally possible thatcombined emitting and receiving means (SE1-SE3, SE4) are used, whichsubsequently carry out the described method in order to determine thedistance to the respective emitter of the pointing pen in each case.

If the pointing pen has two emitters S1 and S2, the distances to emitter1 and emitter 2 can be determined one after the other, or simultaneouslyif non-overlapping frequency ranges are used. The duration of ameasurement, i.e. approx. the modulation period T, is herein dimensionedso short that intermediate movements of the pointing pen are irrelevant.Although it is typical for FMCW measurement that the modulation period Tis much larger than the propagation time of the signal, due to the speedof propagation (speed of light), the short distance (a few metersmaximum) and the frequency range (several GHz; e.g. 85-100 GHz), thevalues are so small that the distance can be determined within theshortest time periods (e.g. <1 millisecond).

For transmission of additional information between the pointing pen andthe medical device, the transmitted signal can also be used. This can bemodulated as desired or modified or supplemented. For example, thesignal of the pointing pen is characteristically interrupted and thusidentifies the pointing pen. To determine the distance, it is notnecessary for the emitted signal to completely contain all frequencies,as illustrated in FIG. 4 .

In this example, the dashed shown signal of the pointing pen comprisessome gaps at the beginning, during which the signal is interrupted, butthe determination of the time offset via the frequency difference of thetwo signal courses only takes place at a later point in time. The gapsin the return signal can encode information, e.g. about the user or thepointing pen. It is also possible to attach information to a modulationperiod, as shown in FIG. 5 .

The data can be modulated onto the carrier signal in any way, forexample as frequency or amplitudes or in a frequency-shift keying (FSK)method. The base frequency is also arbitrary. In the above example, thisis within the spectrum of radar frequencies, which results in the sameemitting apparatus can be used for distance determination and datatransmission. The pointing pen may also have an additional emitter thatis only set up for data transmission. The emission frequency of thisemitter can be selected so that it does not overlap with the radiatedradar waves. This means that the data transmission can also take placeat the same time as the distance measurement.

As already described in the embodiment with laser emitter, in ananalogous manner the pointing pen can be equipped with a push-button andauthentication means. In this case, data transmission takes place asdescribed above via the emitter/receiver means of the pointing pen andthe medical device, which are sensitive to electromagnetic radiation andare not optical sensors.

Embodiment with a Radar Emitter and Angle Sensors

In another embodiment, the pointing pen comprises a single emitter andadditional sensors for determining the inclination of the pointing penin space and the angle that the axis of the pointing pen assumes withthe earth's magnetic field. High-precision sensors are known to thoseskilled in the art for both variables, which are also available insufficiently small versions to be integrated in a pointing pen accordingto the present invention. The integrated 360° angle sensor TLE5012B fromInfineon and the inclination sensor STMicroelectronics IIS2ICLX areexamples.

If the position and orientation in relation to the spatial axes are alsoknown to the medical device (which can be achieved with the aid ofequivalent sensors), the point of impact of the projection of thepointing pen axis can also be clearly determined by emitting thisinformation, as is shown in FIG. 6 .

The angle and/or inclination sensors integrated in the pointing penallow the angles α and β to be unambiguously determined in relation tothe reference axes x, y and z. If this information is transmitted to themedical device and the relative position of the device surface to thesame reference axes is known, a control unit programmed accordingly forthis purpose can calculate the point of impact taking into account theposition of the emitter S1, which has also been determined, for exampleanalogously to the method described above.

Since the relative position of the device surfaces to theabove-mentioned reference axes can change due to translations orrotations of the device or individual device parts that are customary inpractice, in particular due to an inclination of the screen, a dynamiccalculation of the point of impact is at least advantageous. This can beachieved by modelling the device geometry, especially in relation to thereceiver, and linking it to the calculation of the beam geometry. Themodelled device geometry additionally allows the definition of activeinteraction points/surfaces corresponding to device elements that can beselected and influenced. Depending on the user and current deviceoperating status, the active elements can be changed, enablingcontext-adaptive control.

Embodiment with Room/Area Monitoring

In the embodiments described above, the receiver apparatuses arearranged directly on the medical device. However, this is not absolutelynecessary. The receiver apparatuses may be associated with the medicaldevice, or the received signals may be associated with a medical device.For example, the receiver apparatuses may be mounted not as an integralpart of a device to be operated, but within a room or area in whichseveral devices may be located.

For example, receivers can be placed in the corners of the room todetermine the position and pointing axis of the pointing pen in theabove manner. If the room geometry, dimensions, position and orientationof the devices are known, it can be determined which device and whichcomponent is targeted, and control commands can be issued to them via acentral control system. For this purpose, they are networked in terms ofdata technology. Here, too, the pointing pen can be assigned to specificpersons and/or devices to be operated in order to enable or restrictoperation.

This embodiment is particularly advantageous for large, stationaryindustrial plants where the position and orientation of the devices orcomponents to be operated do not change.

This principle may not only be used for operation, but also forinformation acquisition. In this case, information about targeted systemparts is transmitted to a device (tablet, smartphone) that is carriedalong. This can be, for example, operating data, maintenance data orgeneral information. Such a method is particularly advantageous forlarge industrial plants where information about plant components isdisplayed centrally, but information is to be retrieved locally duringoperation.

The pointing apparatus may be integrated into the display apparatuscarried along. For example, a smartphone could be used to point to asystem component. With the aid of corresponding sensors, this can bedetected in the above-mentioned manner, and information about thecorresponding component can subsequently be sent to the smartphone via anetworked central control system.

In certain exemplary embodiments, it is immaterial which method is usedto determine the distance between the emitter and receiver. An amplitudeevaluation of emitted electromagnetic waves of the emitters S1 and S2and triangulation may also be used.

Directional emitting antennas for emitters S1 and S2 may be used, butthis is not a prerequisite.

To avoid unwanted interference from several pointing sticks within aroom, the frequency ranges of emitters S1 and S2 for each pointing pencan be selected so that they do not overlap with those of other pointingsticks. A receiving medical device recognizes the received frequencyrange and selects a corresponding reference signal accordingly. It canbe provided that the received spectrum is divided into correspondingfrequency ranges by signal filters in the medical device. In this way,for example, simultaneous operation of a device with several pointersticks can be realized.

Another way to avoid unwanted interference is to time-multiplex themethod for several pointing pens. This provides for assigning eachpointing pen a periodic time range in which it can emit signals. Thiscan be realized by a central time signal that is emitted and received byeach pointing pen. The pointing pens can then be configured to be activeonly at certain time slots when no other pointing pen is active.

By the methods described, each device will notice whether or not apointing pen is targeting one of its components, which may mean that itis targeting another device. Accordingly, it only becomes “active” inthe first case. In this way, a single pointing pen can be used tocontrol a wide variety of different devices.

While the invention has been illustrated and described in detail in thedrawings and foregoing description, such illustration and descriptionare to be considered illustrative or exemplary and not restrictive. Itwill be understood that changes and modifications may be made by thoseof ordinary skill within the scope of the following claims. Inparticular, the present invention covers further embodiments with anycombination of features from different embodiments described above andbelow. Additionally, statements made herein characterizing the inventionrefer to an embodiment of the invention and not necessarily allembodiments.

While the invention has been illustrated and described in detail in thedrawings and foregoing description, such illustration and descriptionare to be considered illustrative or exemplary and not restrictive. Itwill be understood that changes and modifications may be made by thoseof ordinary skill within the scope of the following claims. Inparticular, the present invention covers further embodiments with anycombination of features from different embodiments described above andbelow. Additionally, statements made herein characterizing the inventionrefer to an embodiment of the invention and not necessarily allembodiments.

1. A system for contactless operation of a device, comprising: anelectromagnetic pointing apparatus, comprising at least oneelectromagnetic signal emitter configured to emit a radar signal; atleast three receivers configured to receive the radar signal emitted bythe at least one electromagnetic signal emitter; and a processorconfigured to: determine a pointing target of the electromagneticpointing apparatus relative to the device based on the received radarsignal; and trigger a function of the device based on the determinedpointing target of the electromagnetic pointing apparatus.
 2. The systemaccording to claim 1, wherein the processor is further configured tocompare the radar signal with a reference signal.
 3. The systemaccording to claim 1, further comprising: an electromagnetic signalemitter associated with the device configured to emit a trigger signalto cause the electromagnetic pointing apparatus to emit the radar signalfrom the at least one electromagnetic signal emitter.
 4. The systemaccording to claim 1, wherein the processor is configured to perform afrequency modulated continuous wave (FMCW) method for determining adistance between the at least one electromagnetic signal emitter of theelectromagnetic pointing apparatus and the at least three receivers. 5.The system according to claim 1, wherein the electromagnetic pointingapparatus further comprises at least a second signal emitter configuredto emit a second radar signal and/or a sensor for determining aninclination of the electromagnetic pointing apparatus; wherein theelectromagnetic pointing apparatus is further configured to determine apointing axis of the electromagnetic pointing apparatus.
 6. The systemaccording to claim 1, wherein the electromagnetic pointing apparatusfurther comprises: at least one push-button; and a modulator configuredto modulate information on the radar signal emitted by the at least oneelectromagnetic signal emitter in response to actuation of the at leastone push-button.
 7. The system according to claim 1, wherein theelectromagnetic pointing apparatus further comprises a fingerprintscanner and/or a communication device configured to communicate with afingerprint sensor wearable under a glove of an operator.
 8. The systemaccording to claim 1, wherein the at least three receivers are arrangedat the device or arranged remotely relative to the device with a definedspatial relationship thereto.
 9. A method for contactless operation of adevice, comprising: emitting, by at least one electromagnetic signalemitter of an electromagnetic pointing apparatus, a radar signal;receiving, by at least three receivers, the radar signal emitted by theat least one electromagnetic signal emitter; determining, by aprocessor, a pointing target of the electromagnetic pointing apparatusrelative to the device based on the received radar signal; andtriggering, by the processor, a function of the device based on thedetermined pointing target of the electromagnetic pointing apparatus.10. The method according to claim 9, further comprising: comparing theradar signal with a reference signal.
 11. The method according to claim9, further comprising: emitting, by an electromagnetic signal emitterassociated with the device, a trigger signal to cause theelectromagnetic pointing apparatus to emit the radar signal from the atleast one electromagnetic signal emitter.
 12. The method according toclaim 9, further comprising: performing, by the processor, a frequencymodulated continuous wave (FMCW) method for determining a distancebetween the at least one electromagnetic signal emitter of theelectromagnetic pointing apparatus and the at least three receivers. 13.The method according to claim 9, wherein the electromagnetic pointingapparatus further comprises at least a second signal emitter configuredto emit a second radar signal and/or a sensor for determining aninclination of the electromagnetic pointing apparatus; and wherein themethod further comprises: determining, by the electromagnetic pointingapparatus, a pointing axis of the electromagnetic pointing apparatus.14. The method according to claim 9, wherein the electromagneticpointing apparatus further comprises at least one push-button and amodulator; and wherein the method further comprises: modulating, by themodulator, information on the radar signal emitted by the at least oneelectromagnetic signal emitter in response to actuation of the at leastone push-button.
 15. The method according to claim 9, wherein theelectromagnetic pointing apparatus further comprises a fingerprintscanner and/or a communication device configured to communicate with afingerprint sensor wearable under a glove of an operator.
 16. The methodaccording to claim 9, wherein the at least three receivers are arrangedat the device or arranged remotely relative to the device with a definedspatial relationship thereto.
 17. One or more non-transitorycomputer-readable mediums having processor-executable instructionsstored thereon for contactless operation of a device, wherein theprocessor-executable instructions, when executed, facilitate performanceof the following: emitting, by at least one electromagnetic signalemitter of an electromagnetic pointing apparatus, a radar signal;receiving, by at least three receivers, the radar signal emitted by theat least one electromagnetic signal emitter; determining, by aprocessor, a pointing target of the electromagnetic pointing apparatusrelative to the device based on the received radar signal; andtriggering, by the processor, a function of the device based on thedetermined pointing target of the electromagnetic pointing apparatus.18. The one or more non-transitory computer-readable mediums accordingto claim 17, wherein the processor-executable instructions, whenexecuted, further facilitate performance of the following: comparing theradar signal with a reference signal.
 19. The one or more non-transitorycomputer-readable mediums according to claim 17, wherein theprocessor-executable instructions, when executed, further facilitateperformance of the following: emitting, by an electromagnetic signalemitter associated with the device, a trigger signal to cause theelectromagnetic pointing apparatus to emit the radar signal from the atleast one electromagnetic signal emitter.
 20. The one or morenon-transitory computer-readable mediums according to claim 17, whereinthe processor-executable instructions, when executed, further facilitateperformance of the following: performing, by the processor, a frequencymodulated continuous wave (FMCW) method for determining a distancebetween the at least one electromagnetic signal emitter of theelectromagnetic pointing apparatus and the at least three receivers.